As GPUs continue to evolve into high performance parallel compute devices, more and more applications are written to perform data parallel computations in GPUs similar to general purpose compute devices. Today, these applications are designed to run on specific GPUs using vendor specific interfaces. Thus, they are not able to leverage CPUs even when both OPUs and CPUs are available in a data processing system nor can they be leveraged across GPUs from different vendors where such an application is running.
However, as more and more CPUs embrace multiple cores to perform data parallel model of computations, more and more processing tasks can be supported by either CPUs and/or GPUs whichever are available. Traditionally, GPUs and CPUs are configured through separate programming environments not compatible with each other. Most GPUs require dedicated programs which are vendor specific. As a result, it is very difficult for an application to leverage both CPUs and GPUs for processing resources, such as GPUs with data parallel computing capabilities together with multi-core CPUs.
Therefore, there is a need in modern data processing systems to overcome the above problems to allow an application to perform a task in any available processing resources, such as CPUs and one or more GPUs, capable of performing the task.